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ABSTRACT 



A Study examined research conducted on the effectiveness of 
different welf are-to-work approaches. Five evaluations begun in the 1980s 
with five-year results indicated that programs focusing on rapid employment 
and job search activities combined with education and training activities 
more often increased employment and earnings and reduced welfare payments, 
when compared with programs that focused solely on job search activities or 
those that placed the greatest emphasis on education. Preliminary results 
(2 -year findings) were obtained from a more recent ongoing evaluation started 
in 1992--the only evaluation designed explicitly to compare the effectiveness 
of a rap id -employment approach with an education-based welf are-to-work 
approach. The results showed that, although each approach has increased 
participants' employment and earnings, neither approach has proven itself 
clearly better. The rapid employment approach did cost about half as much per 
person as the education-based approach. Recent studies of factors affecting 
cognitive development of children in welfare families indicated a mother's 
higher level of educational attainment was one factor that may positively 
affect children's development. A body of research on effects of poverty on 
children's educational attainment suggested a significant positive 
relationship between parents' and their children's educational attainment 
among welfare and non-welfare populations. (Appendixes contain 39 references, 
study scope and methodology, and comments from the Department of Health and 
Human Services.) (YLB) 
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The Personal Responsibility and Work Opportunity Reconciliation Act of 
1996 (P.L. 104-193), enacted in August 1996, significemtly chemged the 
nation’s cash assistemce program for needy families with children. Title I 
of the law replaced the Aid to Families With Dependent Children (afdc) 
cash assistemce program with fixed block grants to states to provide 
Temporary Assistemce for Needy Families (tanf) and ended families’ 
entitlement to assistemce. Several goals of the tanf program are specified 
in the law, including that of ending welfare dependence by promoting 
work over welfare and self-reliance over dependency. Over the years, 
states’ welfare-to-work programs have emphasized different goals and 
philosophies for moving individuals into work and have provided different 
types of services and activities to program participants to help them reach 
those goals. Programs with the goal of rapid employment emphasize quick 
exposure to and entry into the labor force, reflecting the belief that 
participants can best acquire employment-related skills when they are 
working, regardless of the quality of the job. These programs’ service 
strategies tend to rely heavily on job search activities but can make use of 
education and training to some extent. Other programs have the goal of 
skill building, often called an education-beised approach, which usually 
involves a greater initial investment in participants’ education and 
occupational skills, so that when they do enter the labor market, they can 
obtain “good” jobs — those with higher pay, health benefits, and 
opportunity for advancement. The 1996 welfare reform law emphasizes the 
importance of moving welfare recipients into employment and gives states 
greater flexibility to tailor their programs to meet their own goals and 
needs. 

To help it assess how best to assist welfare recipients, the Congress 
directed us in the Higher Education Amendments of 1998 (P.L. 105-244) to 
review research on the effectiveness of various welfare-to-work 
approaches. Specifically, we were asked to examine the research findings 
on (1) the effectiveness of a rapid employment approach, an 
education-based approach (including adult vocational and postsecondary 
education) , and a combination of these two approaches in improving 
employment-related outcomes for welfare recipients and other 
low-income women with children and (2) the effect of welfare recipients’ 
educational attainment, including postsecondary education, on the 
educational attainment of their children. In performing this work, we 
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reviewed studies that had been published from 1988 through mid- 1999 that 
assessed the effectiveness of welfare-to-work approaches by comparing 
outcomes related to employment eind earnings as well as declines in 
welfare payments for those in the programs with those not in the 
programs. Because some welfare-to-work programs may produce results 
in the longer term rather them in the short term, we generally focused on 
evcduations having relatively long-term results (5 years). In addition, 
because many of the welfare recipients who participated in the 
welfare-to-work programs evaluated had less than a high school 
education, the education-based approaches eveduated in the studies we 
assessed primarily provided basic education services rather them 
postsecondary education. As a result, none of the studies evaluated the 
effectiveness of a college education on improving employment-related 
outcomes for welfare recipients. We also identified eind reviewed studies 
about the effect of welfare recipients’ educational attainment on their 
children. (See app. I for a full discussion of our scope and methodology.) 



Results in Brief 



Research conducted to date on the effectiveness of different 
welfare-to-work approaches suggests that programs with a combined 
approach — including both job search assisteince eind some education and 
training — tend to be more effective over a 5-year period than either 
approach alone in increasing employment eind earnings while reducing 
welfare payments. Five evaluations begun in the 1980s with 5-year results 
indicated that programs focusing on rapid employment eind job search 
activities combined with education eind training activities more often 
increased employment eind earnings eind reduced welfare payments, 
compared with programs that focused solely on job search activities or 
those that placed the greatest emphasis on education. In addition, 
preliminary results (2 -year findings) from a more recent ongoing 
evcduation (started in 1992) — the only evaluation designed explicitly to 
compare the effectiveness of a rapid-employment approach with cin 
education-based welfare-to-work approach — found that while each 
approach has increased participeints’ employment eind earnings, so far, 
neither approach has proven clearly better them the other. The rapid 
employment approach did, however, cost about half as much per person 
as the education-based approach. While these studies provide useful 
information, more needs to be known about how well different 
approaches are performing in the current environment created by the 
enactment of welfare reform in 1996, which none of these evaluations 
cover. 
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While research indicates that parents’ educational attainment has a 
positive effect on children’s educational attainment, little information is 
currently available on this relationship specifically within the welfare 
population. Recent studies have identified factors affecting cognitive 
development of children in welfare families. This research, while limited in 
scope, indicates that a mother’s higher level of educational attainment is 
one factor that may positively affect children’s development. In addition, a 
body of research that focuses on the effects of poverty on children’s 
educational attainment suggests a significant positive relationship 
between the educational attainment of parents and their children among 
both the welfare and the nonwelfare populations. 


Background 


AFDC was created by the Social Security Act of 1935 to provide cash 
assistance to families with needy children who had been deprived of the 
support of one of their parents — at that time, mostly children living with 
widowed mothers. The program was not designed to promote 
employment, because, at the time, mothers were generally not expected to 
work outside the home. Over the past several decades, however, the 
public has come to believe that most welfare families should be at least 
partly self-supporting. Efforts to provide education, training, and job 
search assisteince to help welfare recipients prepare for and find Jobs can 
be traced back at least to the 1960s, when the Congress mandated that 
every state operate a Work Incentive (win) program to encourage afdc 
recipients to become self-sufficient, win began primarily as a voluntary 
program focusing onjob search assistance and immediate employment. 
Starting in 1981, WIN demonstration projects were established that gave 
states greater flexibility to design their own programs, and states could 
now require welfare recipients with children aged 6 and older to 
participate. Our reviews of the win program showed that it often served 
those most likely to find employment on their own rather than those less 
job-ready, who needed the most help to become employed. 

The Family Support Act of 1988 eliminated the win program and created 
the Job Opportunities and Basic Skills Training (jobs) program. Research 
conducted before passage of that act showed that welfare recipients were 
a diverse group, making use of the afdc program in different ways. While 
most who used afdc did so for short periods of time, the majority of afdc’s 
resources were devoted to providing benefits to long-term recipients. This 
research also identified several factors that were associated with 
long-term welfare dependence, including recipients’ low level of 
education, single-parent status, higher number of children, disability, and 
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limited work experience.* To better ensure that afdc recipients received 
assistance that would help them avoid long-term welfare dependence, jobs 
required state programs to include a broad range of services, including 
education and training assistance,^ and to provide financial assistance with 
support services such as child care and transportation. The population 
that could be required to participate was changed from those with children 
aged 6 and above to those with children aged 3 and above. In addition, for 
the first time, states were required to place a specified minimum 
percentage of nonexempt welfare recipients in education, training, and 
work-related activities and to target resources to long-term recipients and 
those considered at risk of long-term welfare dependence. 

The Family Support Act also called for an evaluation with a random 
assignment design, which would control for other factors that could affect 
outcomes, to assess the effectiveness of various welfare-to-work 
programs. The Department of Health and Human Services (hhs) , with 
support from the Department of Education, contracted with the 
Manpower Demonstration Research Corporation (mdrc) , a research 
organization that analyzes education- and employment-related programs, 
to conduct this evaluation, which focused on mandatory welfare-to-work 
programs at seven sites. This evaluation is referred to as the National 
Evaluation of Welfare-to-Work Strategies, formerly known as the jobs 
Evaluation. In the mid-1980s, California also contracted with mdrc to 
conduct an evaluation of its state welfare-to-work program, called Greater 
Avenues for Independence (gain) , which became the state’s jobs program 
after 1988. At the beginning of the gain evaluation, California had about 
one-sixth of the nation’s afdc caseload, and gain expended about 
13 percent of federal jobs funds. 

In response to jobs’ increased emphasis on education and training and a 
general belief that these activities could help improve welfare recipients’ 
financial well-being, as they do for the general population, many state and 
local JOBS programs emphasized the provision of education and training. In 
our 1991 report on the implementation of jobs, we found that almost half 
of the states reported a shift from an emphasis on immediate job 



'David Ellwood. Targeting "Would-Be* Long-Term Recipients of AFDC (Princeton, NJ.: Mathematica 
Policy Research, Inc., 1986). 

^Under JOBS, states were to assess the needs and skills of welfare recipients, prepare them for 
employment through education and training as needed, and place them in jobs. Federal rules specified 
certain activities that each state's JOBS program was required to offer. These included education 
activities, job skills training, job-readiness activities, and job development and placement services. 
States also had to offer at least two of the four WIN activities (job search, on-the-job training, work 
supplementation programs, and community work experience programs). Postsecondary education was 
optional under the federal JOBS rules, and states could assign participants to this activity on an 
individual basis. 
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placement under their previous welfare-to-work programs toward a new 
emphasis on long-term education or training.^ In 1995, we reported that 
JOBS participEints nationwide were enrolled in a variety of education eind 
training activities — postsecondary education (17 percent), high school or 
preparation for the general equivalency diploma (16 percent), job skills 
training (13 percent), adult basic or remedial education (7 percent), and 
English as a Second Leinguage training (2 percent).'* 

Several years after the implementation of jobs, our review of state jobs 
programs nationwide showed that only about 1 1 percent of welfare 
recipients were involved in jobs activities each month.® In addition, afdc 
caseloads rose to their highest levels ever, peaking at 5 million families in 
1994. Also during the 1990s, under waivers of the federal rules, several 
states experimented with changes in their afdc and jobs programs. These 
changes included encouraging welfare recipients to work by allowing 
them to keep more of their earnings without losing welfare benefits, 
strengthening and more strongly enforcing work requirements, and 
imposing limits on the length of time a family could receive aid. 

To encourage and facilitate innovation by the states and to address 
continuing concerns among policymakers about growing welfare 
dependence, the Personal Responsibility and Work Opportunity 
Reconciliation Act of 1996, commonly referred to as welfare reform 
legislation, overhauled the nation’s welfare system by abolishing the afdc 
program and establishing tanf block grants. Under tanf, which is 
administered by hhs, states are provided up to $16.8 billion each year 
through 2002 to provide aid to needy families with children. While the 
states have great flexibility to design programs that meet their own goals 
and needs, they must also meet several federal requirements designed to 
emphasize the importance of work and the temporary nature of tanf aid. 
TANF established stronger work requirements for those receiving aid than 
the requirements of its predecessor program, and the population that can 
be required to work now includes all parents, regardless of the ages of 



Welfare to Work: States Begin JOBS, but Fiscal and Other Problems May Impede Their Progress 
{GAO/HRD-91-106. Sept. 27. 1991). 

^Welfare to Work: Participants' Characteristics and Services Provided in JOBS (GAO/HEHS-95-93, 

May 2. 1995). 

^Welfare to Work: Most AFDC Training Programs Not Emphasizing Job Placement (GAO/HEHS-95-1 13, 
May 19. 1995). 
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their children.® In addition, states must enforce a 5-year limit (less at state 
option) on the length of time a family may receive federal tanf assistance.^ 

Our June 1998 report and other studies of tanf implementation show that 
many states and localities have taken steps to transform their welfare 
offices into job placement centers and are encouraging or requiring those 
seeking aid to engage in job search activities as soon as they apply.® Along 
with this increased emphasis on work, welfare offices and workers are 
also focusing more on helping clients address and solve problems that 
interfere with employment. States’ implementation of more work-focused 
programs, undertaken under conditions of strong economic growth, has 
been accompanied by a 45-percent decline in the number of families 
receiving welfare — from a high of about 5 million families in 1994 to 
2.7 million families as of December 1998.® A nationally representative 
survey of families who left welfare from 1995 to 1997 found that 61 percent 
of former welfare recipients were working at the time of the survey, 
although often at low-paying jobs.*® 



^he required minimum participation rate began at 25 percent in fiscal year 1997 and rises to 50 
percent in fiscal year 2002. States receive credit for the degree to which their caseloads have declined 
since fiscal year 1996. While states have considerable flexibility in designing their welfare-to-work 
programs, the legislation prescribes the activities that states may count toward their work 
participation rate. For example, vocational education is limited as a countable work activity both in 
the percentage of recipients who can be engaged in vocational training and count toward the 
participation rate (30 percent) and the length of time a recipient can be in vocational training (up to 12 
months). Moreover, unless states include it in their definition of vocational education training, the 
legislation does not allow postsecondary education to be counted as a work activity toward the states’ 
participation rate. 

^States may exempt from time limits up to 20 percent of those receiving TANF aid and may use their 
own funds to provide aid beyond the federal time limit. 

^Welfare Reform: States Are Restructuring Programs to Reduce Welfare Dependence 
(GAO/HEHS-98-109, June 17, 1998). 

^Welfare Reform: States’ Implementation Progress and Information on Former Recipients 
(GAO/T-HEHS-99-116, May 27. 1999). 

‘“Pamela Loprest, “Families Who Left Welfare: Who Are They and How Are They Doing?" Urban 
Institute Discussion Paper 99-02 (July 1999). See also Welfare Reform: Information on Former 
Recipients* Status (GAO-HEHS-99-48, Apr. 28. 1999). 
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Research Shows That 
a Variety of 
Welfare-to-Work 
Approaches Have 
Positive 

Employment-Related 

Outcomes 



The current research on the relative merits of the rapid employment eind 
education-based approaches does not conclusively show that one 
approach is more effective them the other in increasing welfare recipients’ 
employment eind earnings eind reducing their welfare payments. Of the six 
evaluations of different approaches we identified eind reviewed, five 
evaluations, with 5-year results, covered a range of programs, with some 
combining elements of a rapid employment approach with an 
education-based approach. The results of these evaluations were mixed 
eind, while not conclusive, indicated that programs that combined the 
approaches had more positive effects — and that these effects covered a 
broader cross section of the welfare population — than programs that 
focused more exclusively on providing only job search activities or only 
education. Only one evaluation, part of the National Evaluation of 
Welfare-to-Work Strategies, directly compared the results of a rapid 
employment approach with those of an education-based approach. The 
results for the first 2 years of this ongoing study showed that the outcomes 
were roughly comparable for the two approaches, with both modestly 
increasing participeints’ employment eind earnings eind reducing welfare 
payments. The rapid employment approach was only about half as costly 
per participant as the education approach. None of the studies evaluated 
the effectiveness of a college education in improving employment-related 
outcomes for welfare recipients. All of the evaluations we reviewed 
provided information on welfare-to-work programs operated in the 1980s 
and 1990s; none included results on programs operated since welfare 
reform was enacted in 1996. Currently, hhs is funding 23 studies in 20 
states on welfare reforms that begem under waivers of the afdc program 
but that are continuing in the new welfare environment. These studies will 
provide more information on effective approaches for moving welfare 
recipients into work." 



Little Research Is Available 
That Compares the 
Effectiveness of Various 
Approaches 



We reviewed five evaluations conducted by mdrc that focused on 
mandatory welfare-to-work programs emd for which 5 years of follow-up 
data were available. Because the results of the rapid employment emd 
education-based approaches are expected to unfold in different ways, with 
more immediate results from the rapid employment approach emd more 
delayed results from the education-based approach, we focused only on 
those evaluations with results from 5 years of follow-up. (Other studies we 
identified that did not include 5-year results are listed in the bibliography.) 
Four evaluations were begun in the early 1980s emd covered a variety of 



**For more information, see Web sites http://www.acf.dhhs.gov/programs/opre/rd&e.htm and 
http://aspe.os.dhhs.gov/hsp/hspwelfare.htm 
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programs across the country: the fifth evaluation, begun in the late 1980s, 
included six sites in California. As shown in table 1 , these programs 
generally used some combination of the rapid employment eind 
education-based approaches, although each tended to emphasize one 
approach more than the other. By reindomly assigning welfare recipients to 
different groups — program participeints eind nonparticipeints — evaluators 
C 2 in determine which cheinges in people’s employment, earnings, eind 
welfare payments were due to their participation in the program. This 
reindom assignment method ceinnot tell einalysts which particular aspects 
of the program caused the cheinges or definitively show that the program 
effects were caused by the particular approach used, rather them other 
program features. But such an evaluation cein determine whether the way 
a program was operated at a particular site was effective. 

While evaluations of welfare-to-work programs have been conducted over 
the years, they generally have not been designed to determine the 
effectiveness of one particular approach compared with another. As part 
of the National Evaluation of Welfare-to-Work Strategies, formerly the jobs 
Evaluation, required by the Family Support Act of 1988, mdrc started a 
largely unprecedented effort to compare the effect of two distinct types of 
welfare-to-work approaches — the rapid employment approach versus the 
education-based approach.*^ Under the rapid employment approach, it is 
expected that individuals will move quickly into employment with 
immediate payoffs in increased earnings and welfare savings eind the 
potential to earn more over time. Under the education-based approach, an 
initial investment in education eind training is expected to pay off in the 
future, with increased earnings eind welfare savings once the training is 
completed. This study, the sixth mdrc study that we reviewed, will provide 
up to 5 years of follow-up data and will analyze the effects of each 
approach on a wider array of outcomes, including those for children, in 
the future. Only the results from the first 2 years of this ongoing 
comparison study are currently available. 



comparison evaluation is part of the National Evaluation of Welfare-to-Work Strategies, which 
also includes other components, such as a study, focusing on children who were between the ages of 3 
and 5 when their mothers entered the study, to measure outcomes on children's cognitive development 
and academic achievement, safety and health, problem behavior and emotional well-being, and social 
development. In the national evaluation, over 55,000 individuals at seven sites were randomly assigned 
to groups that remained eligible for specific welfare-to-work programs or to groups that did not 
participate in these programs. In addition to the 2-year findings reported in a study of three sites, 

2-year impacts are available for one site in Portland. Oreg. Also. MDRC has completed evaluations at 
the remaining three sites in Columbus. Ohio; Detroit. Mich,; and Oklahoma City, Okla.; however, these 
studies had not been published at the time of our review. 
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Table 1: Evaluation Studies That 
Assessed the Effectiveness of 
Welfare-to-Work Programs 



Evaluation 


start date° 


Number of programs Basic program 
with specific findings approach 


WlN-era programs 


Baltimore Options 
Program 


1982 


Combined results for 
10 of 18 city welfare 
offices 


Mixed service strategy 


Arkansas WORK 
Program 


1983 


Combined results for 
2 counties 


Rapid employment, 
with a primary focus 
on job search 
activities 


Virginia Employment 
Services Program 


1983 


Combined results for 
11 county welfare 
agencies 


Rapid employment, 
with a primary focus 
on Job search 
activities 


San Diego Saturation 
Work Initiative Model 
(SWIM) 


1985 


Combined results for 
2 of 7 city welfare 
offices 


Mixed service strategy 


JOBS-era program 


California's Greater 
Avenues for 
Independence (GAIN) 
Program 


1988 


Separate results for 
six counties: 
Alameda; Butte; 

Los Angeles; 
Riverside; San Diego; 
and Tulare 


Mixed service 
strategies that 
emphasize rapid 
employment or an 
education-based 
approach to different 
degrees 


National Evaluation of 
Welfare-to-Work 
Strategies: Evaluation 
of Two Welfare-to-Work 
Program Approaches 


1992 


Separate results for 
three sites: 

Atlanta, Ga.; 

Grand Rapids, Mich.; 
Riverside, Calif. 


Comparison of rapid 
employment and 
education-based 
approaches 



“All programs were evaluated for 5 years except for the JOBS-era evaluation of two 
welfare-to-work approaches— 2-year results are available for this ongoing comparison study. 



The MDRC comparison study of two welfare-to-work approaches was 
conducted at three sites — Atlanta, Georgia; Grand Rapids, Michigan; and 
Riverside, California. At each site, welfare recipients were randomly 
assigned to the rapid employment program, the education-based program, 
or a control group. The study compared outcomes for those assigned to 
the program groups with outcomes for those assigned to the control 
group, and compared outcomes for those assigned to one program with 
outcomes for those assigned to the other program. Operating the two 
programs at the three sites simultaneously controlled for the economic 
and programmatic environments that could affect participants’ outcomes. 
Randomly assigning recipients to one of the two programs or a control 
group helped to eliminate any bias from, or effect of, differences in 
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recipients’ characteristics that could affect outcomes. Differences in 
participant outcomes could then be attributed to the program in which 
they participated. In this comparison study and in other evaluations we 
reviewed, not all individuals assigned to a welfare-to-work program group 
necessarily participated in program activities: also, some individuals not 
assigned to a program group sought out and received services at their own 
initiative. In this report, hereafter, we use the term “participants” to refer 
to individuals assigned to a welfare-to-work program group and the term 
“nonparticipants” to refer to those assigned to a control group. 



Evaluations Showed That 
Programs That Combine 
Approaches More Often 
Had Positive Effects Over 5 
Years Than Those That 
Emphasized One Approach 



Although they were not designed to assess the effect of one approach 
compared with another, the four wiN-era and one jOBS-era evaluations 
provide insights into the effectiveness of various approaches. The five 
evaluations covered 10 programs — seven in California — with varying 
approaches for moving welfare recipients into work, and had mixed 
results. Some programs had effects in all three areas — employment, 
earnings, and welfare savings. In some cases, a program increased 
earnings for those considered less job-ready but did not increase earnings 
for those considered job-ready. Although not definitive, the studies show 
that programs combining elements of both the rapid employment and 
education-based approaches — having a rapid employment focus but 
relying on education and training for some participants — tended to have a 
greater effect on participants’ employment and earnings, and on welfare 
savings, than approaches that emphasized just job search activities or just 
longer-term education and training. 



Five-Year Evaluations of Four A study of the 5-year results of four programs operated during the win era 

WIN-Era Programs offers a look at two programs that relied primarily on job search activities 

to move people into jobs in comparison with two programs that provided 
education and training in addition to job search activities (see table 2). The 
Arkansas and Virginia programs are generally considered examples of 
programs focusing on job search only that were relatively low cost and 
provided little in the way of education and training, although some Virginia 
program participants did participate in education and training 
opportunities but participated in such activities only slightly more than the 
control group. The Baltimore Options program featured an initial 
assessment of each participant’s needs and then provided participants a 
choice of activities, swim, in San Diego, was designed to involve as many 
welfare recipients as possible in ongoing activities, following a set 
sequence of activities: job search, work experience, and then education or 
training. 
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Table Emphasis and Other 

of Four WIN-Era Programs Percentage of 

participants who 



Evaluated program 


Program emphasis and 
service strategy 


Net program 
cost per 
person^ 


were ever in 
education and 
training 


Arkansas WORK 


Rapid employment with job 
search first and 3-month 
unpaid work assignments 


$220 


0 


Virginia Employment 
Services 


Rapid employment withjob 
search and some 3-month 
unpaid work assignments 


598 


12 


Baltimore Options 


After initial assessment, 
participants are given some 
choice amongjob search, 
unpaid work assignments, 
or education and training 


1,325 


17 


SWIM 


Involve as many 
participants as possible in 
activities for as long as 
possible, starting withjob 
search, then 3-month 
unpaid work assignment, 
then some education and 
training 


1,212 


24 



“Represents the average cost per participant less the average cost per nonparticipant, adjusted 
to 1993 dollars. Note that net program costs per person are lower for these WIN-era programs 
compared with the JOBS-era programs because welfare recipients with younger children were 
not required to participate in WIN-era programs. Consequently, child care costs were lower. 



The SWIM cind Baltimore Options programs signiflccintly increased total 
earnings for participeints over the 5-year study period, while the earnings 
increases for the Arkcinsas cind Virginia programs were not statistically 
signiflccint. Arkansas WORK did, however, have signiflccint savings in 
welfare payments.'^ As shown in figure 1, only the swim program both 
produced welfare savings and increased participeint earnings. Over the 5 
years, the swim program increased total earnings of participcints more them 
$2,000 above the total earnings increase for nonparticipcints eind reduced 
welfare payments almost $2,000 per participant. On the basis of a detailed 
cinalysis of these studies’ findings, researchers concluded that the 
Baltimore Options program was the only program that helped some 



*^For our interpretations, we used a common signiHcance level of 5 percent (.05) or less, which was 
stricter than that used in some evaluations. Total earnings impact represents the difference between 
the total average earnings between the program participants and nonparticipants. Welfare savings 
represent the differences between the total average welfare payments for participants and 
nonparticipants and do not take into account the costs of the program. In averaging the earnings or 
savings, those individuals with a zero value were included. The average earnings of only those with 
earnings would be higher. 
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participants find higher-paying jobs than the jobs nonparticipants found. 
The programs that relied primarily on job search generally resulted in 
those participants who would have worked eventually beginning to work 
sooner, and motivated some participants who would not have worked to 
get a job. For the near term, this resulted in participants relying more on 
their own earnings than on welfare, although it did not increase their 
overall financial well-being or increase their earnings capacity.*^ 



Figure 1 : Effect on Total Earnings and ■■ 
Welfare Savings per Program 2 soo 

Participant Over a 5-Year Study Period 
for Four WIN-Era Programs 
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Note: Earnings differences between program participants and nonparticipants were statistically 
significant only for Baltimore Options and SWIM; welfare savings were statistically significant only 
for Arkansas WORK and SWIM. 



Five-Year Evaluation of Six In 1988, mdrc began a 5-year evaluation of California’s gain program in six 

JOBS-Era Programs in counties. This six-site evaluation affords an opportunity to examine the 

California performance of programs that varied in key ways, including program 

emphasis and service strategy. While all of the county programs used a 
mixed service strategy, including job search activities and education and 
training, the extent to which they emphasized rapid employment or 
longer-term education and training varied, as shown in table 3. The study 



BEST COPY AVAILABLE 



^^Daniel Friedlander and Gary Burtless, Five Years After: The Long-Term Effects of Welfare-to-Work 
Programs (New York, N.Y.: Russell Sage Foundation. 1995). ' 
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researchers noted that, among the six counties, the Riverside program 
promoted the strongest message about the importance of moving quickly 
into employment. The other counties’ programs placed more emphasis on 
skill building, with the Alameda, Los Angeles, and Tulare programs placing 
participants in education and training longer than the other counties. 

The county programs and environments differed in other key ways that 
can affect the operation and success of a welfare-to-work program. For 
example, as shown in table 3, some counties had much higher percentages 
of welfare recipients generally considered hard to employ (or less 
job-ready) than other counties, as measured by the percentage of program 
participants assessed by each program to be in need of bcisic education. A 
key feature of the gain program at the time of the evaluation in these six 
counties was its emphasis on adult basic education. Depending on 
participants’ need for basic education, gain placed participants into groups 
that were considered more job-ready and less job-ready, gain assigned the 
job-ready participants to job search initially and, in general, assigned the 
less job-ready to basic education, although sometimes these participants 
were given the option to look for work or to look for work and participate 
in basic education concurrently. As shown in table 3, the percentage of 
participants who were enrolled in education and training ranged from 28 
to 53 percent. 
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Table 3: Program Emphasis and Other Features of the Six GAIN County Programs 

Participants 

assessed to need Participants enrolled 



County program 


Program emphasis and 
service strategy 


Net program cost 
per person^ 


adult basic 
education 
(percentage) 


in education and 
training activities 
(percentage) 


Special features 


Alameda 


Emphasized longer-term 
education and training; 
used mixed service 
strategy 


$5,597 


65 


53 


Targeted to 
long-term welfare 
recipients only 


Butte 


Placed more emphasis 
on education and 
training than on rapid 
employment; used mixed 
service strategy 


2,904 


49 


28 


Used more 
intensive case 
management than 
other programs 


Los Angeles 


Emphasized longer-term 
education and training; 
used mixed service 
strategy 


5,789 


81 


44 


Targeted to 
long-term welfare 
recipients only 


Riverside 


Stressed rapid 
employment but 
included short-term 
education; used mixed 
service strategy 


1,597 


60 


36 


None 


San Diego 


Placed more emphasis 
on education and 
training than on rapid 
employment; used mixed 
service strategy 


1,912 


56 


37 


None 


Tulare 


Emphasized longer-term 
education and training; 
used mixed service 
strategy 


2,734 


65 


49 


None 



Note: The information on these programs generally represents the programs and the status of 
their clients at the end of 3 years. The programs changed to some extent over the 5-year period. 
For example, some of the original nonparticipants were allowed to participate in some JOBS 
activities. 



® Represents the average cost per participant less the average cost per nonparticipant, adjusted 
to 1993 dollars. 



The six programs had mixed results for participants in increasing 
employment, earnings, and welfare savings, compared with 
nonparticipants. Of the sites with statistically significant employment 
increases, three — Los Angeles, Riverside, and San Diego — increased the 
percentage of participants who had been employed sometime during the 5 
years by at least 5 percentage points above the percentage for 
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nonparticipcints. (See fig. 2.) The Riverside program had the greatest 
effect: the percentage of those who had held a job during the 5-yecir study 
period was 10 percentage points higher for participeints them for 
nonparticipcints (72 percent versus 62 percent). 



Figure 2: Employment Rates for 
Program Participants and 
Nonparticipants Over the 5-Year Study 
Period in Six GAIN Counties 
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Note: Employment rate differences between program participants and nonparticipants were 
statistically significant only in Los Angeles, Riverside, and San Diego. 



Two of the programs that had statistically significcint effects on the 
employment of participcints compared with nonpcirticipcints — Riverside 
cind Scin Diego — also had significcint effects on earnings for their 
participcints (see fig. 3) . The Butte program did not result in increased 
employment but did result in increased earnings. For the programs with 
statistically significcint differences, the total effect on earnings per 
participcint compared with nonparticipcints over the 5 years reinged from 
almost $3,000 in Scin Diego to just over $5,000 in Riverside. Four 
programs — Alameda, Los Angeles, Riverside, cind Scin Diego — significcintly 
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reduced welfare payments; these savings ranged from about $1,400 to 
about $2,700 per participant (see fig. 4). 



Figure 3: Total Earnings of Program 
Participants and Nonparticipants Over 
the 5-Year Study Period in Six GAIN 
Counties 
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Note: Differences in earnings were statistically significant between program participants and 
nonparticipants only in Butte, Riverside, and San Diego. 
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Figure 4: Average Total Welfare 
Payments for Program Participants 
and Nonparticipants Over the 5-Year 
Study Period in Six GAIN Counties 
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Note: Differences in welfare payments between program participants and non participants were 
statistically significant only in Alameda, Los Angeles, Riverside, and San Diego. 



Because of the importance of understanding how a welfare-to-work 
program works for different segments of the welfare population, the gain 
evaluation also looked at results for two key subgroups: participants and 
nonparticipants who were job-ready (those determined not to be in need 
of basic education) and participants and nonparticipants who were less 
job-ready (those determined to be in need of education). Job-ready 
participants in three programs had statistically significant higher earnings 
or lower welfare payments when compared with job-ready 
nonparticipants, with the Riverside and San Diego programs having both 
higher earnings and lower welfare payments. These two programs had 
relatively large effects on earnings — almost $6,000 and more than $5,000, 
respectively — for the job-ready group. 

For those considered less job-ready, three programs — Butte, Riverside, 
and Tulare — had statistically significant increases in earnings, as shown in 
figure 5, with the effect on earnings ranging from about $2,700 in Tulare to 
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more than $5,000 in Butte. Five of the programs had significeint welfare 
savings, as shown in figure 6. Two programs — Butte and Riverside — had 
both higher participeint earnings eind welfare savings. The Butte program 
performed relatively well for the less job-ready with earnings of 
participants $5,000 higher than those of nonparticipeints eind 
per-participeint welfare savings of about $3,900. While the Butte, Riverside, 
cind Tulare programs all had a statistically significeint effect on participeint 
earnings for those needing basic education, the effects occurred later in 
the study period in the Butte and Tulcire programs them in the Riverside 
program. This demonstrates that some progreuns that do not produce 
results in a short time frame may do so in the longer term. 

Among the six programs, participeints determined in need of basic 
education were generally assigned to adult basic education activities.’® 

The programs generally increased participation in these activities for 
participeints compared with nonparticipeints, but a majority of those who 
were in need of basic education did not obtain a general equivalency 
diploma. For those programs that had higher earnings eind welfare savings, 
it is unclear the extent to which provision of basic education contributed 
to these effects.’® 



*^In the Riverside program, all participants — whether they needed education or not — were strongly 
encouraged to start with job search as their first activity. 

*®For more information on adult education for welfare recipients, see Edward Pauly, The JOBS 
Evaluation: Adult Education for People on AFDC: A Synthesis of Research (Washington, D.C,: 
Department of Education and HHS, Dec. 1995). Also see Janet Quint. The JOBS Evaluation: Educating 
Welfare Recipients for Employment and Empowerment: Case Studies of Promising Programs 
(Washington, D.C.: Department of Education and HHS, 1997). 
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Figure 5: Total Earnings for Program 
Participants and Nonparticipants 
Determined to Be In Need of Basic 
Education Over the 5-Year Study 
Period in Six GAIN Counties 
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Note: Differences between program participants and non participants were statistically significant 
only in Butte. Riverside, and Tulare. 
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Figure 6: Welfare Payments to 
Program Participants and 
Nonparticipants Determined to Be in 
Need of Basic Education Over the 
5-Year Study Period in Six GAIN 
Counties 
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Note: Differences between program participants and nonparticipants were statistically significant 
in Alameda, Butte, Los Angeles, Riverside, and San Diego. 



Among the six programs, only the Riverside and San Diego programs had 
statistically significant effects in all three areas — total employment, 
earnings, and welfare savings — over the 5 years. In addition, the Riverside 
program, the least costly, achieved results in earnings and welfare savings 
for both the job-ready and less job-ready participants. The study 
researchers attributed the success of the Riverside program to a 
combination of factors: conveying a strong, consistent message about the 
importance of quick employment for participants, even for those who 
began the program with education and training; relying on a mixed 
strategy including job search, education and training, and other activities 
and services: enforcing participation requirements: devoting some staff to 
job development activities to help identify employment opportunities for 
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participeints; £ind focusing staff on results by, for example, using 
performzmce stemdards to measure their performzmce.’^ 

The Riverside program is considered one of the most successful 
large-scale mandatory welfare-to-work programs on the basis of its 
significant effect on a rzmge of outcomes zmd the fact that it reaches a 
broad cross section of the welfare caseload. Even so, the program still did 
not end participzmts’ dependence on welfare or lead to employment at 
wages above poverty levels for mzmy families. The 3-year results showed 
that, after entering the program, 41 percent of participants were still 
receiving welfare and 81 percent had income at or below the poverty line. 
The percentage of those still receiving welfare is no doubt affected to 
some extent by the relatively high maximum earnings levels for afdc in 
California at the time of the study. For example, in California in fiscal year 
1991, a three-person family would have needed monthly earnings of $959 
to become ineligible for afdc.’® In a state with a relatively high earnings 
limit such as California, many more welfare recipients are likely to 
combine work and welfare, while in states with lower earnings limits, any 
job, even at the minimum wage, can result in a family’s moving off welfare. 



MDRC Comparison Study 
Shows That Results From 
Rapid Employment and 
Education-Based 
Approaches Did Not Differ 
in the Short Term 



In the ongoing comparison study conducted at three sites (Atlanta, Grand 
Rapids, and Riverside) by mdrc as part of the National Evaluation of 
Welfare-to-Work Strategies, the welfare-to-work programs implemented at 
the study sites were designed to provide “pure” examples of the rapid 
employment and education-based approaches. In the rapid employment 
approach, though, the program could assign participants to education and 
training activities if they were not able to find a job during the initial job 
search period. 



The rapid employment approach emphasizes quick job placement, 
assuming that work habits and skills are better learned at a job than in a 
classroom and that any job can be a stepping-stone to a better one. 
Program staff only briefly assess participants assigned to this program 
before they attend a 3- to 5-week job club. The job club includes classroom 



*^The GAIN Riverside program discussed here differs somewhat from the Riverside program currently 
being operated as a “pure" rapid employment approach as part of the MDRC comparison evaluation 
under way. The researchers noted that the GAIN Riverside program emphasized a broader range of 
activities for participants, placing more of them in education and training than the Riverside rapid 
employment approach in place now. 

*®We estimated that, in fiscal year 1991, the monthly amount of earnings needed for a family to no 
longer be eligible for AFDC varied widely across the states, from a low of $385 to a high of $1,1 11, with 
a median of $632. See Self Sufficiency: Opportunities and Disincentives on the Road to Economic 
Independence (GAO/HRD-93-23, Aug. 6, 1993). 
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instruction on how to look for a job eind provides supervised job search. 
Participeints who have not found a job at the end of this period are more 
fully assessed by staff to see what activities and services would help them 
get a job. Depending on that assessment, participemts are assigned to more 
time in job club, individual job search, or short-term (up to 9 months) 
basic education, vocational training, or work experience. A participant 
who completes such an assignment without finding a job is assessed again 
and assigned to the same or emother of these activities.*® 

The education-based approach assumes that participants need to invest 
some time in education or training before seeking employment so that 
they can acquire skills that will help them get good jobs and thus leave 
welfare permanently. Participants assigned to this approach first undergo 
a detailed assessment by program staff to determine their job-related skills 
and interests and to identify potential barriers that they face to getting 
employment. Depending on the assessment, these participants are 
assigned to up to 2 years of basic education, vocational training, college, 
or work experience. A participant who remains unemployed after 
completing the assignment is reassessed and assigned to the same or 
another of these activities, a job club, or an individual job search.^® 

The 2-year earnings and welfare payment outcomes of both the rapid 
employment and education-based program participants were significantly 
better than outcomes for nonparticipants, but the outcomes of the 
participants in the two approaches were not different enough from each 
other to conclude that, overall, one approach is more effective than the 
other. In Atlanta and Grand Rapids, there was no statistically significant 
difference in the total earnings effects between the two program 
approaches.^* Among participants with a high school diploma or 



‘®During the study’s 2*year follow-up period. 41 percent (Riverside) to 69 percent (Adanta) of 
participants assigned to the rapid employment approach participated in job search and 8 percent 
(Riverside) to 31 percent (Grand Rapids) participated in an education or training activity. 

^“There were some differences between the intended and actual sequence of program activities and 
emphases. According to the researchers, assessments were generally not in-depth. For example, only 
one program had an up-front assessment that was longer than a few hours — the program in Grand 
Rapids, using an education-based approach, had an up-front assessment that lasted a week in a 
classroom setting. In addition, at all three education-based sites, basic education was by far the most 
commonly assigned first program activity, followed by vocational training. Assignments to work 
experience or college were very rare. During the study’s 2-year follow-up period. 47 percent 
(Riverside) to 58 percent (Grand Rapids) of participants assigned to the education-based approach 
participated in an education or training activity and 12 percent (Atlanta and Riverside) to 14 percent 
(Grand Rapids) of recipients participated in job search. 

^'A comparison of the outcomes of the full sample of the two groups can be made only in Atlanta and 
Grand Rapids, because in Riverside the education-based approach was available only to participants 
who did not have a high school diploma or equivalent or who achieved relatively low scores on basic 
skills tests administered at orientation. 
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equivalent, there were no differences between the two approaches in 
terms of total earnings. The rapid employment approach did, however, 
show greater effects on earnings them the education-based approach 
among participemts in Gremd Rapids who did not have a high school 
diploma or equivalent (42 to 44 percent of the total sample at the three 
sites). In Grand Rapids, the rapid employment approach produced 
significcmtly greater savings in toted welfere payments them the 
education-based approach, but that difference was not statistically 
significant in Atlemta. 

While both approaches produced positive outcomes, they did so at very 
different program costs. Across the three sites, the average per-participemt 
cost for the rapid employment approach (above what was spent on 
nonparticipcmts) was $1,550. The average per-participant cost for the 
education-based approach was $3,077, nearly twice the cost of the rapid 
employment approach. 



The Effect of Welfare 
Mothers’ Educational 
Attainment on Their 
Children’s 
Educational 
Attainment Is 
Currently Unknown 



While research indicates that parents’ educational attainment is positively 
related to children’s educational attainment, little information is currently 
available on this relationship specifically within the welfare population. 
Recent studies have identified factors affecting the cognitive development 
of children in welfare families. This research, while limited in scope, 
indicates that one factor that may positively affect children’s development 
is the level of their mothers’ educational attainment. This issue has also 
been emalyzed within a body of research focusing on the effects of poverty 
on children’s educational attainment. In these studies, which did not 
sample welfare recipients exclusively, emalysis has included measuring 
and controlling for welfare receipt in order to determine whether the 
welfare population is different from the general population. In general, 
these research results show a significemt positive relationship between the 
educational attainment of parents and their children among both the 
welfare and nonwelfare populations. 



Findings from research that focuses on the development of the children 
within the welfare population are inconsistent. One longitudinal study of 
614 children whose families received afdc found that mothers’ prior 
education corresponded to higher reading scores but not to higher math or 
vocabulary scores for children. This study found no significcmt effect 



^^Hirokazu Yoshikawa, “Welfare Dynamics, Support Services. Mothers’ Earnings, and Child Cognitive 
Development: Implications for Contemporary Welfare Reform," Child Development . Vol. 70. No. 3 
(May/June 1999). The study examined mothers’ education level 1 year before the birth of the child in 
the study. 
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over the study period from a mother’s increasing her education over the 
first 5 years of her child’s life. As part of the National Evaluation of 
Welfare-to-Work Strategies currently being conducted, researchers will 
identify how the welfare-to-work experiences of mothers affect their 
children.^^ A preliminary descriptive report on preschool-aged children in 
this evaluation found that a child’s development was associated with the 
mother’s educational attainment.^'* Future reports from this evaluation will 
include school data for children approximately 5 years later. In addition to 
this evaluation, several states have included a component in evaluations of 
their welfare programs — partially funded by HHS — to look at the effects of 
various welfare reforms on children, including children’s school 
achievement.^* Although these studies will not measure the completion of 
schooling for these children, they will provide additional information on 
how mothers’ educational attainment affects the progress of their 
children’s education. 

Other applicable research, which has not analyzed welfare recipients 
exclusively, has attempted to determine whether and how the welfare 
population differs from the general population in terms of children’s 
educational attainment. These studies show that a parent’s various 
circumstances, such as economic status and educational level, have a 
significant effect on children’s educational attainment across the sample, 
even when analysis controls for welfare receipt. In all of the studies that 
provided detailed results, mothers’ educational attainment consistently 
had a significant effect on the educational attainment of their children. 
These findings suggest that a significant relationship between a mother’s 
and her children’s educational attainment may also hold true for the 
welfare population. 



Concluding 

Observations 



Our review of research conducted over the past two decades shows that a 
welfare-to-work approach with a strong employment focus can have 
positive effects on participant earnings and employment and on welfare 
costs. However, we do not yet definitively know, especially in the long 
term, whether a rapid employment or education-based approach works 
best for increasing the employment and earnings of welfare recipients and 
reducing their dependence on welfare. Future results from an ongoing 




^’The three sites for this study are Atlanta, Ga.; Grand Rapids. Mich.; and Riverside. Calif. This study is 
contained within MDRC comparison study referred to earlier that compares the rapid employment and 
education-based welfare-to-work approaches. 

^^Kristin A. Moore and others, The JOBS Evaluation: How Well Are They Faring, AFDC Families With 
Preschool-Aged Children in Atlanta at the Outset of the JOBS Evaluation (Washington, D.C.: HHS, 
Office of the Assistant Secretary for Planning and Evaluation. 1995). 

“States include Connecticut, Iowa, Minnesota, Indiana, and Florida. 
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Study designed specifically to compare the effectiveness of a rapid 
employment approach with that of an education-based approach may shed 
more light on this issue. In the meantime, our review of 10 evaluated 
programs that had 5-year results indicates that those welfare-to-work 
programs that combine elements of both approaches — emphasizing rapid 
employment but tailoring services to some extent to meet the differing 
needs of welfare recipients — may best meet the goals of increasing 
employment and earnings for welfare recipients while at the same time 
reducing welfare payments. These types of programs can play an 
important role in moving welfare recipients into the labor force and 
increasing the extent to which they rely on their own earnings rather than 
government aid. Nevertheless, even the most successful program in 
Riverside did not usually lead families to higher-paying jobs or move them 
out of poverty during the time period studied. Future research will need to 
focus on longer-term program outcomes, the effect of the 1996 welfare 
reform legislation, and what works best for particularly hard-to-employ 
populations. 



Agency Comments 
and Our Evaluation 



We obtained comments on a draft of this report from HHS, which stated 
that, overall, it concurred with our findings. More specifically, hhs agreed 
that, of the welfare-to-work approaches that have been tested, programs 
with a combined approach — emphasizing rapid employment but providing 
education and training when appropriate — appear to be most effective. 

HHS commended us in our selection of rigorous studies using random 
assignment; however, hhs noted that this criterion eliminated any studies 
regarding postsecondary education, since no rigorous evaluations have 
been done of the effectiveness of such programs for welfare recipients. 

HHS made several suggestions to further clarify information contained in 
the report. First, HHS pointed out that the costs for the wiN-era programs 
would naturally be lower than the costs for the jOBS-era programs because 
welfare recipients with younger children were exempt from participating 
in the earlier programs and, consequently, costs for child care would be 
lower. We agree with this suggestion and have included it in the report. 
Second, hhs noted that while we focused on effects on cumulative 
earnings over the 5-year follow-up period, which is a good measure of the 
overall impact of a program, the effects on earnings at the end of the 
follow-up period might be a better indicator of future earnings. We agree 
with HHS that assessing the effects on earnings at the end of the follow-up 
period can provide important information. However, our analysis of the 
results of the six-county gain evaluation showed that the only counties 
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that showed statistically significcint increases in earnings for the last year 
of the follow-up period (Butte, Riverside, and Sem Diego) also showed 
significemt increases over the 5-year follow-up period; as a result, we did 
not report separately on the effects at the end of the study time period. 
Furthermore, we noted that for participemts considered to be less 
job-ready, effects occurred later in the study period for several counties, 
indicating that some programs that do not produce results in a short time 
frame may do so in the longer term. Finally, regarding the section about 
the effect of welfare mothers’ educational attainment on their children’s 
educational attainment, hhs noted that the report should more clearly 
explain how studies documented a mother’s educational background. We 
have made revisions to this section to clarify whether the education level 
was documented at the onset of a study or during the study period. 
However, additional information was not available on the percentage of 
mothers in a study sample with increases in education during the study 
period. In addition, HHS provided technical comments, which we 
incorporated in the report where appropriate. HHS’ comments are included 
in appendix II. 

We provided a draft of this report for technical review to the Departments 
of Education and Labor; we also provided a copy to two experts in welfare 
issues. The Department of Education and the experts said we had 
accurately characterized the research available in the field. They also 
provided technical comments, which we incorporated as appropriate. The 
Department of Labor had no comments. 



We are sending copies of this report to the Honorable Donna E. Shalala, 
Secretary of Health and Human Services; the Honorable Alexis M. Herman, 
Secretary of Labor; the Honorable Richard W. Riley, Secretary of 
Education; and state tanf directors. We will also make copies available to 
others upon request. 

If you or your staffs have any questions about this report, please call me 
on (202) 512-7215. Other GAO contacts and staff acknowledgments are 
listed in appendix III. 
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Cynthia M. Fagnoni 
Director, Education, Workforce, emd 
Income Security Issues 

Page 26 



GAO/HEHS-99-179 Welfare-to-Work Approaches 



B-282174 



List of Addressees 



The Honorable William V. Roth, Jr. 

Chairmcin 

The Honorable Dciniel Patrick Moynihcin 
Ranking Minority Member 
Committee on Finemce 
United States Senate 

The Honorable James M. Jeffords 
Chairman 

The Honorable Edward M. Kennedy 
Ranking Minority Member 

Committee on Health, Education, Labor, and Pensions 
United States Senate 

The Honorable William F. Goodling 
Chairman 

The Honorable William L. Clay 

Rcmking Minority Member 

Committee on Education emd the Workforce 

House of Representatives 

The Honorable Bill Archer 
Chairmcm 

The Honorable Charles B. Rangel 
Rcmking Minority Member 
Committee on Ways emd Meems 
House of Representatives 



O 



Page 27 



29 



GAO/HEHS-99-179 Welfare-to-Work Approaches 



Contents 



Letter 




1 


Appendix I 
Scope and 
Methodology 




30 


Appendix II 
Comments From the 
Department of Health 
and Human Services 




32 


Appendix III 
GAO Contacts and 
Staff 

Acknowledgments 




35 


Bibliography 




36 


Related GAO Products 




44 


Tables 


Table 1: Evaluation Studies That Assessed the Effectiveness of 
Welfare-to-Work Programs 


9 




Table 2: Emphasis and Other Features of Four WIN-Era Programs 


11 




Table 3: Program Emphasis and Other Features of the Six GAIN 
County Programs 


14 


Figures 


Figure 1: Effect on Total Earnings and Welfare Savings per 
Program Participant Over a 5-Year Study Period for Four 
WIN-Era Programs 


12 




Figure 2: Employment Rates for Program Participants and 
Nonparticipants Over the 5-Year Study Period in Six GAIN 
Counties 


15 




Figure 3: Total Earnings of Program Participants and 

Nonparticipants Over the 5-Year Study Period in Six GAIN 
Counties 


16 



o 



Page 28 



GAO/HEHS-99-179 Welfare-to-Work Approaches 



Contents 



Figure 4: Average Total Welfare Payments for Program 17 

Participants and Nonparticipants Over the 5 -Year Study Period in 
Six GAIN Counties 

Figure 5: Total Earnings for Program Participants eind 19 

Nonparticipants Determined to Be in Need of Basic Education 
Over the 5-Year Study Period in Six GAIN Counties 
Figure 6: Welfare Payments to Program Participants and 20 

Nonparticipants Determined to Be in Need of Basic Education 
Over the 5-Year Study Period in Six GAIN Counties 



Abbreviations 

AFDC Aid to Families With Dependent Children 
GAIN Greater Avenues for Independence 
HHS Department of Health and Human Services 

JOBS Job Opportunities and Basic Skills Training 
MDRC Manpower Demonstration Research Corporation 

SWIM Saturation Work Initiative Model 

TANF Temporary Assistance for Needy Families 

WIN Work Incentive 



Page 29 



GAO/HEHS-99‘179 Welfare-to-Work Approaches 



Appendix I 

Scope and Methodology 



To address the mandated objectives, we identified relevant evaluation 
studies of welfare-to-work programs that help welfare recipients and other 
low-income women with children become employed. To be included in our 
review, evaluations had to meet the following criteria: 

• A program could have started before 1988, but its evaluation had to have 
been published since 1988, after the passage of the Family Support Act. 

• A study had to measure the effect of welfare-to-work approaches on 
employment-related outcomes such as employment, earnings, and welfare 
payments. 

• A study had to rigorously evaluate the program by controlling for factors 
that could affect employment-related outcomes. 

To identify the relevant evaluations as well as identify information on the 
impact of welfare recipients’ educational attainment on the educational 
attainment of their children, we searched several on-line bibliographic 
databases. These databases included Sociological Abstracts, Social 
SciSearch, ERIC, the Welfare Information Network, and ECONLIT. We 
also reviewed bibliographies of research studies on these issues and 
consulted with experts on welfare-to-work issues to identify other studies 
we should consider. We met with officials at the Departments of Health 
and Human Services, Labor, and Education to obtain further information 
on pertinent evaluations. 

We identified only one evaluation that compared the effectiveness of a 
rapid employment approach with that of an education-based approach and 
included this evaluation in our review. In selecting other welfare-to-work 
evaluations for review, we included only evaluations with impacts for 
follow-up periods of at least 5 years. Consequently, we identified for 
review six evaluations, which evaluated a total of 13 programs. Because 
many of the welfare recipients who participated in the welfare-to-work 
programs evaluated had less than a high school education, the 
education-based approaches evaluated in the studies we assessed 
primarily provided basic education services rather than postsecondary 
education. As a result, none of the studies evaluated the effectiveness of a 
college education on improving employment-related outcomes for welfare 
recipients. All the evaluations used research designs that controlled for 
other factors that could affect outcomes. For example, participants were 
randomly assigned to either a program group, which was subject to the 
program being evaluated, or to a control group, which continued under a 
previous program or no program. The experience of the control group 
members — who, at their initiative, could use services elsewhere in the 
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community— indicates what would have happened to the program groups 
in the absence of special intervention, providing a benchmark for 
measuring program effects. The principal outcomes measured in the 
evcduations were employment, earnings, and welfare savings. For each 
outcome in each study, the researchers had compared results for the 
participants receiving program services with those for participants in the 
control group and identified statistically significant differences that were 
deemed to be program impacts or effects. The evaluation reports 
estimated the likelihood that these differences occurred by chance by 
using standard tests of statistical significance. We did not independently 
verify the information in the evaluation reports. 

We conducted our work in accordance with generally accepted 
government auditing standards between February 1999 and July 1999. 
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Comments From the Department of Health 
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DEPARTMENT OF HEALTH & HUMAN SERVICES 



Office of Inspector General 



Washington, O.C. 20201 



btP 2 1999 



Ms. Cynthia M. Fagnoni 
Director, Education, Workforce, and 
Income Security Issues 
United States General 
Accounting Office 
Washington, D.C. 20548 

Dear Ms . Fagnoni : 

The Department has carefully reviewed your draft report entitled, 
"Welfare Reform: Assessing the Effectiveness of Various Welfare- 

to-Work Approaches." The comments represent the tentative 
position of the Department and are subject to reevaluation when 
the final version of this report is received. 

The Department also provided some technical comments directly to 
your staff. 

The Department appreciates the opportunity to comment on this 
draft report before its publication. 



Enclosure 



The Office of Inspector General (OIG) is trainsmitting the 
Department's response to this draft report in our capacity as 
the Department's designated focal point and coordinator for 
General Accounting Office reports. The OIG has not conducted 
an independent assessment of these comments and therefore 
expresses no opinion on them. 



Sincerely, 





June Gibbs Brown 
Inspector General 
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Comments of the Department of Health and Human Services on the U S. General Accounting 
Office’s Draft Report, Welfare Reform: Assessing the Effectiveness of Various Welfare-to-Work 
Approaches (GAO/HEHS-99- 1 79) 

We have reviewed the General Accounting Office (GAO) draft report and have the following 
comments. On the whole, we were pleased with both the approach and the conclusions of the 
report. In particular, we agree that the research results available are not definitive but indicate 
that, of the welfare-to-work approaches which have been tested, the most effective is probably 
one that emphasizes rapid employment but also provides education and training when 
appropriate based on the differing needs of welfare recipients. 

Also, the fact that the report cites findings from random-assignment experiments as evidence of 
the relative effectiveness of various approaches to welfare employment programs is 
commendable and reflects the sizeable numbers of experimental design evaluations that have 
been done in this area. Where they can be mounted, rigorous experimental evaluations provide 
more reliable measures of program impacts than quasi- and non-experimental evaluations, which 
are subject to selection bias and invalid counterfactual measures. Given that the report seeks to 
determine what approach has the greatest long-term success, it is hard to fault the choice of 
studies used. The report examines all experimental studies that have results for 5 years of 
follow-up. In addition, 2-year follow-up data from the National Evaluation of Welfare-to-Work 
Strategies sites that directly compare Labor Force Attachment and Human Capital Development 
approaches are cited. 

At the same time, the commitment to consider only random-assignment studies means that this 
report is unable to address the questions posed specifically with respect to the effectiveness of 
post-secondary education. There has not been a rigorous evaluation of the effectiveness of such 
programs for welfare recipients. As a result, this report may disappoint some of the requestors. 
The GAO may wish to explwn this in the discussion of scope and methodology. 

We also have a few comments on the use of these studies. First, while the report acknowledges 
the difficulty of comparisons across sites and across time, it does not appear to note that costs are 
generally higher for programs that include parents of pre-school-age children than those that are 
restricted to parents of older children, due nuunly to higher child care costs. Thus the costs for 
older programs that studied participants whose youngest child was 6 or older, such as the 
Baltimore Options Program or the San Diego Saturation Work Initiative Model, will not be 
comparable to those for later Job Opportunities and Basic Skills Training programs, such as 
California’s Greater Avenues for Independence Program, which included participants with 
younger children, even after adjusting for inflation. 

Second, the report focuses on the impacts on cumulative earnings over the follow-up period. 
While this is a good measure of the overall impact of the program, the difference in earnings at 
the end of the follow-up period may be a better indicator of the future trajectory of impacts. 

Third, the section on the effect of maternal education on child outcomes is not always clear on 
when studies are looking at the initial difference in a mother's educational background versus 
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when they are looking at the difference that an actual change makes—which only an experiment 
that provides an educational intervention can test. For example, in the discussion of the 
Yoshikawa study, it would be helpful to indicate what percentage of the mothers in the sample 
had increases in education during the period of the study. If the percentage of mothers with 
increases in education was very small, this is not a good test of whether such increases have an 
effect on a child’s educational development. 
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